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researcher's NS assistants. Role plays were transcribed and coded for 
three nonverbal behaviors, head nods, gaze direction changes, and 
gestures. Descriptive statistics, including reliability and standard 
error of measurement for the proficiency tests were calculated for 
the two tests and three nonverbal behaviors in role plays 2 and 3. A 
Significant difference was found for head nods in role play 4. A set 
of scales for assessing nonverbal ability was constructed on the 
basis of the quantitative and qualitative analysis of the nonverbal 
behaviors in the role plays. Results of the study indicate that it is 
possible to define degrees of nonverbal ability. (Author/JL) 
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Abstract 

This study attempts to design an instrument for assessing the 
nonverbal ability of foreign language learners. The subjects are 
educated Japanese non-native speakers of English (n=28) and 
educated North American native speakers of English {n=20) for 
baseline data. Materials used were the institutional TOEFL for 
linguistic proficiency, the SPEAK test for oral proficiency, and a 
series of four role plays for the collection of nonverbal data. 
Proficiency tests were administered to NNS subjects who were paired 
according to the results of the TOEFL. NNS and NS subjects 
performed the three role plays in pairs and the fourth with the 
researcher's NS assistants. Role plays were transcribed and coded 
for three nonverbal behaviors, head nods, gaze direction changes, 
and gestures. Descriptive statistics, including xeliability and 
standard error of measurement for the proficiency tests, were 
calcujates for the two tests and three nonverbal behaviors in role 
plays 2-3. A MANOVA was performed for the main effect Group for 
the three nonverbal behaviors in three role plays. A significant 
difference was found for head nods in role play 4. A set of scales 
for assessing nonverbal ability was constructed on the basis of the 
quantitative and qualitative analysis of the nonverbal behaviors in 
the role plays. The results of this study indicate that it is 


possible to define degrees of nonverbal ability. 
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Assessing the Nonverbal Abilit 
of Foreign Language Learners 

The accurate assessment of orai proficiency in a foreign 
language can be crucial for non-native speakers of English whe need 
favorable evaluations for generai academic purposes at American 
universities, for employment, to work as teaching assistants at 
American universities, or for other purposes which require a 
specified level of English proficiency. A variety of scales and 
testing metnods such as the Foreign Service Institute (FSI) 
interview and its derivative ACTFL/ETS gnidelines have been created 
to assess oral proficiency but these have also been criticized for 
their narrowness as norm-referenced tests (Bachman, 1990) or for 
being no more than discrete-point tests without a basis in theory 
or research (Bachman, 1988; Bachman & Savignon, 1986). 

With the expansicn of our framework for communicative 
competence (Canale & Swain, 1980; Canale, 1983), there has been a 
need for improving oral proficiency evaluation using the 
communicative paradigm. Along these lines, Bachman and Palmer 
developed an oral interview test (Bacnman & Palmer, 1983) in an 
attempt to account for more of the reliable variance in the 
learner's oral proficiency than existing measures (Bachman & 
Palmer, 1982) in terms of what Bachman (1988, 1490) refers to as 
Communicative Language Ability (CLA). 


However, even Bachman and Palmer's test relies solely on the 
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vocal channel for its evaluation of oral proficiency in spite of 
recent calls to pay more attention to nonverbal behavior as an 
integral part of speech (von Raffler-Engel, i980; Baird, 1983) anā 
Clear indications that nonverbal behaviors are part of the 
scciolinguistic and strategic competence components of 
conmununicalive Competence (Canale, 1983; Canale & Swain, 1980). 

In general, language testing experts have neglected nonverbal 
behavior. Bachman's (1990) major work on language testing does not 
include nonverbal behavior under sociolinguistic competence and it 
notes that strategic competence definitions 

include non-verbal manifestations of strategic 

competence, which are clearly an important part of 

strategic competence in communication, but which 

will not be dealt with in this book. (p. 100) 

Problems which may be related to intercultural differences in 
the use of nonverbal behaviors can, however, affect the rating of a 


second language learner's oral language proficiency causing them to 





be rated either higher or lower than their actual language 
proficiency level (Nambiar and Goon, 1993; Neu, 1990). As Baird 
(1983) puts it, 
...the importance of nonverbal behavior in overall 
communication effectiveness is obvious, and the 
difficulties in assessing the skills involved 


should not blind us to their significance. (p. 33) 
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Assessing Nonverbal Ability 


Nonverbal communication (NVC} is commonly considered 
“communication effected by means other than words" (Knapp & Hall, 
1992). It has also been defined as the influencing of another 
person by means of faciai expression, tone of voice, gaze, 
gestures, posture, bodily contact, spacial behavior, clothes or 
appearance, nonverbal vocalizations, or smell (Argyle, 1966). 

These nonverbal behaviors are also often interpreted through verbal 
signs, an indication of the importance of viewing verbal and 
nonverbal communication as cpposite ends of a continuum with some 
overlap between the two (Knapp & Hall, 1988). Thus, while they may 
be independent at the extremes of the continuum, there are 
instances where they are meaningfully integrated. 

It is useful, then, to consider the role of NVC in terms of 
this relationship with spoken language. Nonverbal vocalizations, 
gestures, facial expressions, and gaze are integrated with words 
and can "amplify and disambiguate them" (Argyle, 1988, p.106). 
Until as recently as ten years ago, however, there were no studies 
which dealt with the interactional nature of NVC (Kenaon, 1984). 

Gestures, for example, serve a number of roles in relation to 
speech, replacing it or alternating with it in a variety of ways 
(Kendon, 1984). nies and gaze behavior serve to regulate 
interactions as nonverbal backchanneling signals or turn-taking 
Signals. Nonverbal communication also plays a part in making up 


for the inability to communicate something verbally due to 
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linguistic deficiencies, as in strategic competence (Canale, 1983). 
While NVC can be described separately from spoken language and 
often has specific meaning independent of ianguage (Ekman, 1976), 
it is the integration of language with nonverbal behavior that is 
most important for assessing language learners' proficiency. 
Kendon (1984) and McNeiii (1965) both note that nonverbal behavior 
such as gestures shares the planning process with speech and thus 
is not really completely nonverbal. In other words, rather than 
being a separate system of communication, NVC is viewed here as an 
integral part of verbal communication. 

The appropriate use of some nonverbal behaviors (in 
particular gestures, head nods, and gaze, which ameliorate the 
interaction process as backchanneling or turn-taking signals) can 
also be included under textual competence in Bachman's (1990) CLA 
model as "ways in which interlocutors organize and perform the 
turns in conversational discourse..." (p. 88). Edmondson (1981) 
also notes the importance of NVC in discourse in his observation 
that eye contact, for example, may be more important than speech 
for successful conversational development. The use of nonverbal 
behavior may even provide a key to the learner's proficiency level 
since, as Ferch, Haastrup, and Phillipson (1984) point out, 

...the activation of strategic competence presupposes 

an inability to make use of parts of linguistic or 


pragmatic competence. (p.168) 
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The level of use cf NVC for strategic, compensatory purposes may 
provide a clue te the learner's level of proficiency with greater 
use indicating lower or inadequate linguistic proficiency. 

Appropriate use of NVC will also include such issues as its 
synchrony with the speaker's own speech and with that of the 
lutecioculor, which can be seen in terms of Blalystok's control 
dimension (Bialystok, 1999; Bialystok & Sharwood Smith, 1935). It 
is expected that lower oral proficiency learners will have poorer 
synchrony or coordination due to their lower ability, affecting the 
coordination of the verbal and nonverbal planning mentioned above 
(Kendon, 1984; McNeill, 1985). As Færch and Kasper (1984) note, 

...advanced learners, who are capable of planning 

longer units, can often predict a communicative 

problem well in advance and attempt to solve it 

beforehand, as part of the normal planning 

process. (p.60-61) 
Any planning deficiency at lower levels then could also be 
reflected in the learner's use of nonverbal behaviors. 
Frequent pauses accompanied by diverted gaze by a language 
learner could be interpreted as a sign of lower proficiency 
when the actual content of the discourse is such that it 
would not place an exce:sive burden on the average learner's 
oral proficiency. 


The testing of nonverbal ability in conjunction with 
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linguistic and oral proficiency should concentrate on 
nonverbal features that are closely integrated with speech. 
The Communication Competency Assessment Instrument (CCATI) 
which was developed to assess the basic communication skills 
necessary for success at an American college (reported in 
Rubin, 1982) includes “use of nonverba! signs appropriate 
for the situation" (p. 21) as one of 19 competencies needed 
in an educational context. Developed as an alternative 
method for assessing academic skills, the CCAI included 
assessment of the ahility to recognize and use appropriate 
gestures, eye contact, and facial expressions in academic 
situations. 

Generally speaking, any scale for evaluating the nonverbal 
benavior of speakers of English as a foreign language should also 
consider gestures Meds head movement) and gaze direction. 
Such a scale should therefore include some of the most important 
functions of NVC such as backchanneling, which is often used 


differently by native English speakers and people from other 





cultures (Maynard, 1987; Maynard, 1990; White, 1989). Japanese, 
for example, use head movement as a nonverbal backchannel signal 
more often and differently from Americans when speaking Japanese as 
well as in intercultural discourse in English (Maynard, 1990). 
Nonverbal behavior can also be described in terms of textual 


use, which involves the appropriate or acceptable use of nonverbal 
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behaviors to manage Lurns in conversational discourse, and 
strategic use, whick involves compensatory behavior to make ur for 
deficiencies in linguistic proficiency or otherwise clarify verbal 
messages. The three types of nonverbal behavior under the two 
categories of textual and strategic use can be used to assess 
nonverbal proficiency in conjunction with tests of oral 
proficiency: 


TEXTUAL USE 


Gestures 
° Hands are used by the speaker to emphasize speech. 


. Vertical head movement (nod) is used as a backchannel signal 
by the listener to indicate attention, O E or 
agreement. 

° Vertical head movement (nod) is used by the speaker as a 
within-turn or turn-end signal. 

° Horizontal head movement (shake) is used by the listener to 


indicate disagreement or to accompany laughter. 


Gaze 

. Listener-directed gaze is used at the end of an utterarce to 
elicit a backchannel response. 

. Terminal gaze (prolonged gaze starting just before the end of 
an utterance) is used to signal the end of the utterance. 

. Speaker-directed gaze is used to signal attentiveness. 


Facial Expressions 
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° Smiles are used to indicate attention cr agreement. 

. Frowns are used to indicate disagreement or lack of 
una cstanding. 

STRATEGIC USE 

Gestures 

° Mime (hand gestures) is used to compensate for a linguistic 
deficiency such as the lack of & necessary lexical item. 

° Hand gestures are used to support spoken language to 
communicate spacial relationships and physical shapes which 
are not always easily understood using spoken language alone. 
This iist of nonverbal behaviors is limited to hand gestures, 

head movement (nods), gaze direction in relation Lo the 

interlocutor, and facial expressions as behaviors which can be 
easily observed and rated by a language teacher. Tt is not, 
therefore, meant to be a complete list of nonverbal behaviors, but 
rather a short list of behaviors in which differences may he 
readily observed and may vary as a result of differences in the 
oral proficiency level or cultural background of the learner. 

Furthermore, while such a list gives the impression that these are 

discrete behaviors, there is a degree of integration among the 

nonverbal behaviors themselves. Nonverbal backchanneling, for 
example, may include a nod at the listener accompanied by 
listener-directed gaze. This list is meant to simplify and focus 


the rating task by grouping behaviors according to their role in 
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The purpose cf this study, then, is to develop scales for 
assessing foreign language leerners' nonverbal ability as a 
component of their oral proficiency. In order to develop a method 
for assessing nonverbal ability in a English as a foreign language, 


the following research Questions wili be addressed. 


p 


What nonverbal behaviors (such as gestures, gaze, and facial 


expressions) do native speakers (NS) and non-native speakers 


2. How do NNS nonverbal behaviors differ from those of NSs in 
Similar situations? 
3. If NNSs do exhibit different nonverbal behaviors from NSs, to 
what extent is this related to oral L2 proficiency? 
4. To what extent is it possible to identify degrees of 
nonverbal proficiency? 
Method 
Subjects 
The subjects in this study are 28 non-native speakers and 20 
native speakers of English. The non-native speakers are educated 
middle-class Japanese male and female EFL learners consisting of 24 
students, two faculty members, and.two office staff members of a 
Japanese university. The native speakers are 20 educated white 
middle-class North American maies and females who are EFL teachers 


or graduate students in TESOL at a Japanese branch cf an American 
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university in Japan. They provide the baseline data for nonverbal 
behaviors. 

The Japanese subjects were recruited using a poster placed on 
a university bulletin board and also directly from among the 
researcher's students, the office staff, and tne faculty in 
exchange for a free adminislralion of Lhe TOEFL. While a ranuon 
sample would be desirable, this sample is ecologically valid since 
those who are required to take oral proficiency tests generally 
self-select into the programs or situations which require the 
tests. While such a sample presents potential problems for 
statistical analysis, the subjects do reflect the type of person 
who would actually be tested in this situation. 
Instruments 

Instruments used in this study are the Test of English as a 
Foreign Language (TOEFL) as a general English proficiency measure, 
the Test of Spoken English (TSF) as a measure of oral proficiency, 
and a series of role plays developed by this researcher to elicit 
data for the development of nonverbal ability (NOVA) scales. 
TOEFL 

The Institutional Testing Program (ITP) version of the Test 
of English as a Foreign Language was used to essess overall English 
proficiency because of its generally high reliability of r=.95 and 
validity. The TOEFL meets the criteria of content validity for the 


university setting of this study with its academic content, and as 
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a test of linguistic competence it is appropriate for its construct 
validity. Its high correlations with other measures of English 
proficiency (ETS, 1932) gives it a high degree of criterion-relatea 
validity. 

TSE 

Ocal proficiency was measured with Test Form 1 of the 
Speaking Proficiency English Assessment kıt (SPEAK) version of the 
Test of Spoken English (TSE), a "semidirect" measure of oral 
proficiency which rates pronunciation, grammar, fluency, and 
overali comprehensibility. Interrater reliability of the TSF has 
been shown to range from .87 to .92 for the four categories of 
ratings (ETS, 1990). Content validity is met since the TSE was 
develuped so that the items chosen would reflect performance on 
similar items on the TOEFL and Foreign Service Institute (FS1) oral 
proficiency interview. Criterion-related validity is supported by 
studies which have shown it to correlate well with the FSI rating 
of oral proficiency at .79 (ETS, 1990). 

Role Plays . 

Nonverbal data was elicited using four role plays performed 
in English (two repeated in Japanese as explained below in 
Procedures). The first role play between two friends was used as a 
warm-up and to reduce the reactivity effect of the video cameras. 
Since some subjects had never performed role plays before, this 


informal format was easier for them to begin with. The following 
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two role plays were between unequals in an interview format {a 
language school interview and a part-time job interview) since many 
oral proficiency tests have an interview format in which the 
interviewer is superior te the person interviewed. This NNS/NNS 
format also made it possible to compare these interactions with 
NS/NS interactions as well as NS/NWS interactions. In the fourth 
role play, the subject was required to make a request of a teacher 
for a letter of recommendation (see Appendix A for English versions 
of the role plays}. 

The role plays were in a format similar to the simulations 
used by Haastrup (1986) as part of what is called a "structured 
oral incerview" (p. 76) for assessing oral proficiency. In this 
simulation, pupils were given an instruction card telling them that 
they were <pplying for a job and have been called for an interview. 
The cards also indicated in the pupils' native language what 
information would be required. This kind of test is widely used in 
Denmark to test English language students after five years of 
English instruction. 

The role play format has content validity as a format which 
is used in oral proficiency testing (Bachman & Palmer, 1983; 
Haastrup, 1986) and to elicit data for discourse analysis 
(Edmondson, 1981; Kasper 1981), and it is expected to elicit the 
most natural NVC that can be expected in a testing situation. 


Video recordings of the subjects performince the role plays in 
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English and Japanese {for Japanese subjects) were used te ānāiyze 
nonverbal behavior and to provide the basis for the cevelopment of 
the scale for measuring nonverbal ability. 
Procedures 

The English versions of the role plays in the Apnendix A were 
piloted with native speakers and Japanese university students. As 
a result cf the initial piloting, it was decided to have the 
subjects sit facing each other with a table between them and to 
take the role play cards away from the subjects before beginning 
because frecvent referral to them inhibited the naturalness of the 
conversations as well as the nonverbal behavior itself. The 
contents were then revised and transieted into Japanese so that the 
subjects' understanding of the role plays would not depend on their 
English proficiency and so that the language they used would come 
from their own English ability and not from the cards. This would 
also insure that the strategic use of NVC would be elicited for 
unknown vocabulary. Subjects performing the pilot role plays, for 
example, did not know how to say "letter of recommendation" for 
role play 4. Almost all of them mimed the shape of a piece of 
paper while explaining what they needed. 

The Japanese translations were checked by a Japanese English 
teacher and piloted with another pair of Japanese university 
students. In this piloting, the subjects repeated the two 


interview role plays in Japanese after performing the final English 
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rele play individually with a native speaker. This pilcting showec 
that subjects could easily perform the role plays in the allotted 
time and would not need another warm-up role play in Japanese 
before the Japanese performances. 

The institutional TOEFL was administered to the subjects 
Logelher in a classroom al their university by an experienced TOEFL 
supervisor. The SPEAK Form 1 version of the TSE was administered 
individually to each subject in the researcher's office according 
to the instructions supplied with the test. The researcher 
remained in the room to observe them and assist if there were any 
problems. The recordings of these tests were rated b; two 
experienced native speaker EFL teachers according to the TSE 
guidelines. 

The subjects were paired according to their TOEFL scores to 
insure that there would be as equal a linguistic burden as possible 
between subjects when performing the role plays. They performed the 
role plays in the researcher's office, where they also took the 
TSE, seated facing each other with a small table betvieen them. 
Their performances were recorded using two video camcorders placed 
unobtrusively pointed at less than 45 degree angles toward each 
subject. After the subjects were seated, they were given the 
Japanese versions of the role play instructions shown in Appendix 
A. After signifying that they understood, they were given the role 


play cards for role play 1 (warm-up role play) between two friends. 
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The subjects were given four minutes to perform this and subsequent 
role plays and did have access to the cards during the 
Sey EEN 

After performing role play 2 (a language school interview) 
and role piay 3 (a part-time job interview), which were designed to 
elicit eguivalentl dala from both subjects by giving each a chance 
to play the status~-higher role of the interviewer, each subject 
performed role play 4 (asking for a letter of recommendation) with 
the researcher's native speaker assistant who was an experienced 
EFI teacher. Finally, the subjects replayed role plays 2 and 3 in 
Japanese to enable the comparison for the subsequent study of their 
nonverbal beaavior when speaking Japanese with their performances 
aes speaking English. As previously stated, there was no warm-up 
for the Japanese performances. 

Of the 15 NNS pairs originally performing the role plays, one 
subject was not included in the analysis because he was Chinese and 
another was excluded because his glasses prevented a clear view of 
his gaze direction. Their partners were included in the 28 
subjects analyzed here. 

The English versions of the role plays were performed as 
above with the 20 North American NS subjects to obtain baseline 
data on nonverbal behavior by NSs in similar situations. These 
were performed at facilities of their university and recorded using 


two video camcorders placed as with the NNSs. Another American NS 
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assistant performed role play 4 with each of the subjects. 
Additional instructions for the NS assistants are included in 
Appendix A. 

The role plays were then transcribed using the discourse 
transcription (DT) method of Du Bois, Schuetze-Copurn, Cumming, ana 
Paolino (1993). Based on Lhe review of the nonverbal behaviors 
made during this transcription, nods, gaze direction, and gestures 
ware chosen for analysis and coded according to a simple 
transcription system ceveloped by the author and shown in Appendix 
B. Only the middle three minutes of each role play were coded for 
analysis to eliminate greetings and formulaic speech at the 
beginning and to assure that none of the pairs finished too early. 

In this system, nods were coded as nodding occasions, meaning 
the performance of one or more nods as one nonverbal signal. 
Nodding occasions included eight of Maynard's (1989) 10 contextual 
categories of vertical head movement: as a backchannel 
continuer/pause filler with or without verbalization, 45 a 
continuer signal by the listener to the speaker, as a clause 
boundary or turn-end signal occurring simultaneously with speech, 
as a turn transition claimer or transition filler during pause 
only, with expressions of affirmation or agreement such as "yeah" 
or “uhuh," for emphasis, and as a preturn claim. Rhythmic nodding 
with speech was not counted. 


Gaze direction was labeled as "toward" when the subject was 
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perceived to change gaze direction toward the interlocutor. It was 
labeled as "away" when when the subject was perceived to change 
gaze direction away from the interlocutor. Head direction was also 
used as a measure of gaze direction (Duncan & Fiske, 1985). For 
the purpose of analysis, only the number of changes in gaze 
direction toward the interlocutor were counted. 

Gestures labeled included miming to compensate for linguistic 
deficiencies and gestures to support or enhance speech by 
indicating direction or location, spaciai relationships, counting, 
shapes, and size. Gesturing for emphasis was not included. A 
brief description of the movement and meaning of the gesture was 
included in parentheses after the code for gestures. 

Analyses 

The dependent variable in this study is nonverbal behavior. 
The independent variables are linguistic proficiency measured by 
the TOEFL and ora] proficiency measured by the TSE. The contro] 
variable is nationality with only Japanese subjects used for rating 
nonverbal ability. Potential moderator variables are sex, age, and 
years of English study. 

Nonverbal Data 

Nonverbal data was analyzed by observing the role play videos 
of the NNS subjects and NS subjects and noting similarities and 
differences among the NNS subjects' in the performance of the three 


nonverbal behaviors listed and comparing NNS and NS pérformances in 


Dd 
Oo 


Se rere a nl 








(< 
ERIC 





Assessing Nonverbal Ability 


20 
English. The three types of nonverbal behaviors which show 
potential for rating nonverbal ability were counted and a 
multivariate analysis of variance (MANOVA) was carried out to 
determine if there was a significant difference between the mean 
number of occasions for the NNS and NS subjects' nonverbal 
behaviors in each of Lhe role plays. The resulls fur rele plays 2 


and 3 were averaged for the purpose of analysis resuiting in three 
nonverbal scores for each type of interaction: NNS/NNS and NNS/NS 
assistant for Japanese subjects and NS/NS and NS/NS assistant for 
baseline data subjects. This averaging was carried out to 
compensate for differences which may have been a result of the 
alternating roles in role plays 2 and 3. Although gaze direction 
while listening has been found to be consistent among members of 
the same culture in spite of individual differences (La France & 
Mayo, 1978), it may be affected by the speaker's identity (Kendon & 
Cook, 1969), and the amount of listening itself is affected by 
whether the higher- or the lower-status role is being played. The 
role play results for the Japanese language interactions are not 
included in this study. 
Statistical Analysis 

Data consist of interval scores from the TOEFL, the TSE 
ratings of pronunciation, grammar, fluency, and overall 
comprehensibility, and the counts of nonverbal behaviors in role 


plays, as well as reliability coefficients for proficiency tests. 
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Reliability 

Reliability is an important measure for assuring the 
consistency of testing instruments. Theoretically, high 
reliability means that test takers can be expected tc receive 
Similar scores on repeated administrations of the test. Lower 
reliability means a grealer range of difference in possible scores 
for such repeatec administrations. Reliability for the TOEFL for 
these subjects was calculated using Cronbach's alpha coefficient 
for internal consistency. interrater reliability for the TSE was 
calculated using Pearson correlation corrected with the 
Spearman-Brown prophecy formula used as suggested by Brown (1992) 
for decision reliability. 
Validity 

While reliability is an important quality for a good test, it 
is of no use without validity, which is the extent to which a test 
actually measures what it claims to be measuring. As previously 
stated, the validity of the TOEFL and TSE are weil established. 
They appear to measure what they claim to be measuring and 
correlate well with other measures of the same constructs, 
establishing criterion-related reliability. 

Nonverbal data collection is based on descriptions of 
nonverbal behaviors found in the literature and supported by 
baseline data gathered from native speaker subjects, thus 


demonstrating content validity. 
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Other statistics 





Other statistics will include descriptive statistics (mean, 
range, standard deviation, and standard error of measurement) for 
the TSE ratings and TOEFL scores. 

Results 

Table 1 shows the descriptive statistics for the TOEFL and 
SPEAK test. As expected, the results of these tests are generally 
normally distributed and reliable measures of proficiency with the 
exception of the SPEAK grammar section which includes only two 
scores for grammar, making it a poor measure for lower proficiency 
subjects who may score zero on one or both parts of this section. 

Insert Tables 1 and 2 


and Figure 1 about here 





Table 2 shows the descriptive statistics for the number of 
occasions of head nodding, gaze toward the interlocutor, and 
meaningful gestures in role plays. The distributions are generally 
normal, with the exception of gesture distributions which is 
positively skewed. At least one subject in each type of 
interaction performed a number of gestures that was well above the 
mean. Figure i shows the mean number of occasions graphically. 
There is a clear difference in the means for all behaviors in each 


role play with a consistently greater number of NNS nods in NNS/NNS 
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and NNS/NS interactions and a greater number of gaze direction 
changes toward the interiocutor and gestures in NS/NS interactions. 

Since the MANOVA for the Group main effect was found to be 
Significant, F(6, 41) = £.412 for the Pillais, Hotellings, and 
Wilks' lambda tests of significance at p < .C001, univariate ANOVAS 
were explored for each nonverbal behavior in each type of 
interaction. The effect for Group was significant (F=27.604; 
df=1/46; p<.0001) for head nodding in role play 4, a finding 
consistent with Maynard's (1987, 1990). 

Discussion 

Several of the findings here provide possible insights which 
contribute to the preliminary drafting of a set of scales for 
assessing nonverbal ability. There is, however, a significant 
difference between a nonverbal rating scale and one which 
concentrates on oral proficiency. In the case of oral proficiency, 
the rating will concentrate on production, as in the case of the 
SPEAK test used here, while leaving the measurement of receptive 
skills to written tests such as the TOEFL. Nonverbal 
communication, on the other hand, is unique in that it covers both 
reception and production in a given interaction and it is not 
always accompanied by speech. 

As for what behaviors NNSs and NSs use in similar situations, 
this study found through quantitative and qualitative analysis that 


vertical head nodding, gaze direction, and hand gestures would be 
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the most revealing behaviors because they are easy to observe and 
interpret and because they occur frequently even in a limited 
three-minute period for both NS and NNS interactions. 

In addition, NNS behaviors are different from those of NS 
subjects in this study. The number of NNS nods, as to be 
expected, were significantly grealer in WNS/NS inleracLions and 
slightly greater even for NNS/NNS interaction, though not 
significant. What makes this a very ratable behavior is not only 
the frequency but also the nature of the occasions in which they 
occur. This culturally-based difference can be seen in differences 
in backchanneling (Maynard, 1987, 1989, 1990: White, 1989), as 
follows: 


(1) (Role Play 4 - Request for Recommendation, NS4A)! 


Professor: How long will it take you? 
Student: um, 
A 
sE figure, 
two years. 
T N 
Professor: Probably. 


(2) (kole Play 4 - Request for Recommendation, NNS13A) 


Professor: these are all important things. 


Student: Uhuh. 
N 
Professor: uh, 


Maybe you could write down just, 
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AE 
ay 


your basic, 
A 
uh, 


your schooling, 
N 
and, 
Coy (Role Play 4 - Request for Recommendation, NNS12B) 
Prefessor: when do you want me, 
to write the letter. 
A 
...next week, 
T N 


Or, 


next [month]} 


Student: jah], 
Yeah. 
um, 
A 
..-Next month, 
T N 
[and], 
A 
Professor: [By next] month. 


The NS in example (1) had approximately the average number 
of head nods for baseline data subjects. This subject had no 
examples of backchannel head nodding. Most nodding was for 
emphasis or as a turu-end signal, as in this example. The NNS 


subject in example (2) was a high oral } roficiency subject with a 





less than the average number of head nods. This subject had 


examples of turn-end nods, and nods with expressions of affirmation 
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cr agreement such as "yeah" or "ubuh," which occur frequently in 
NNS data. The NNS subject in exarple (3) was a low oral 
proficiency subject who had slightly more than the average amount 
of head nodding for NNS subjects. There was frequent nodding as a 
packchannel/continuer signal without verbalizaticn while listening 
es well as wilh afficmelion or agreemenl. Of course, the oral data 
itself did differ between proficient and iess proficient subjects. 
NS and higher oral proficiency subjects spent less time listening 
and more time talking in role play 4 than lower oral proficiency 
NNSs because they understood what the professor was asking them. 

As for gaze direction, while there are similarities between 
NS and NNS subjects for frequency cf changes and certain types of 
usage, there are greater differences between higher and lower oral 
proficiency subjects. Examples (1) and (3) above illustrate how 
both NS and NNS speakers precede the answer to a question with a 
question with a filled pause, "um," accompanied by a gaze change 
away. Usually the next gaze change toward comes at or near the 
beginning of the answer. Lower oral proficien~y subjects often 
delay their gaze change toward the interlocutor, as follows: 


(4) (Role Play 4 - Request for Recommendation, NNO15B) 


Professor: Do you have any hobbies. 
Did you belong to a club. 
A 


Student: Ty 


I belong to, 
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.table tennis club. 
T N 
‘rofessor: Yhuk. 


In America they call table tennis ping pong. 

This lower proficiency subject looks away at the end of the 
revhrased question by the researcher's NS assistant and does not 
change direction toward the interlocutor until the end of the 
answer. ‘This could be related to control variability (Bialystok & 
Sharwood Smith, 1985), affecting the synchrony of gaze direction 
anca speech due to the learner's uncertainty over what form of 
languace is appropriate. Gaze away is usually a sign of thinking, 
therefore prolonged gaze away may be sign of the learner's lower 
proficiency, and frequent and long occasions of gaze away could be 
rated as a sign of "lower" nonverbal ability. 

Differences between NSs and NNSs for hand gesturing to 
enhance speech or to compensate for linguistic deficiency cannot be 
interpreted only on the basis of simple counting and comparisons of 
means. Content and success are also important factors. 

NSs did not necessarily use hand gestures. Two out of 20 
baseline deta subjects did not gesture in any of the role plays as 
compared to three out of 28 NNS subjects. The two major 
differences between NSs and NNSs, aside from compensatory usage by 
Ss, was the variety and type of gestures used. Frequency itself 


is not an indicator. One NNS used hand gestures pointing at his 
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own chest nine times to indicate "I," "my," or "me" out of a total 
of 11 hand gesturing occasions. The other gestures were pointing 


at the NS assistant for "you" and holding up four fingers while 


saying "four years." While NSs sometimes pointed at their chest 
for "I," the NNSs overuse could be considered inappropriate by 
native speaker norms. Interestingly, although il was expected that 


167) 
oO 
rh 
D 
i 
= 


some of the NNSs would point at their noses for "me" as i 
encountered in Japan, none of these subjects did. 

One lower proficiency subject used hand gestures 18 times, 
four times for "me" and twice for "you." The rest of the gestures 
were for direction or miming the shape of a “letter of 
recommendation” or the act of writirg. It is interesting that the 
researcher had earlier thought that this subject was more 
proficient, until very low scores on the TOEFL and SPEAK test 
suggested otherwise. The subject's use of gaze direction and hcad 
nodding also seemed to be more appropriate by native speaker 
standards. 

Success or failure at an attempt to use mime as a 
compensatory strategy can also be seen as an indicator of the 
learner's level of nonverbal ability, as follows: 


(5) (Role Play 3 - Convenience Store, NNS6A) 
Student: um, 


.. What situation. 
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Manager: um, 
A 
...(6)<L2 nan to iu L2> ((What do you say?)) 
G(mimes carrying something with both hands) 


Student: Ca- carry. 
Manager: Carry. 
T N 


Puring a long pause, the subject playing the manager Carnot think 
of a lexical item. Before reverting to code switching, she begins 
tc mime the word that she cannot think of. The subject accepts the 
interlocutor's interpretation, indicating that the gesture for 
compensatory purposes was successful. If the interlocutor had not 
supplied an acceptable word, this example would be considered a 
failure. 

Thus, while the performance of some kinds of nonverbal 
behavior are related to language proficiency, there are exceptions 
to this, as Neu (1990) has pointed out. This supports the idea 
that nonverbal ability can be treated as a separate construct from 
linguistic proficiency. For testing purposes, nonverbai ability 


can be integrated into Bachman's (1990) CLA mcdel as shown in 





Figure 2. 
Insert Figure 2 about here 
As a result of the quantitative and qualitative analysis 


carried out in this study, a set of scales for assessing nonverbal 
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ability can be constructed as shown in Table 3. 





insert Tabie 3 about here 





Under textual ability, looking away and infrequent nodding 
are associated with lower textual ability for frequency, and 
packchanneling frequentiy and at every pause such as Japanese 
speakers do would be associated with lower textual appropriateness. 
cnder strategic ability, compensatory usage will be rated according 
to need to use and successful performance, while the rating for 
appropriateness would depend on how often gestures are used and the 
content of the gesturing. The subject who gestured for “me” 
freauently and occasionally pointed at his or her nose would be 
rated lower for appropriateness. 

Conclusion 

This study has investigated the possibility of assessing the 
nonverbal ability of foreign language learners. It has 
demonstrated that: 

i. Vertical head nods, gaze direction changes, and gestures are 
nonverbal behaviors that are used meaningfully by both NSs 


and NNSS in simi ir situations. 


N) 


These ncnverbal behaviors differ in frequency, in their 
synchrony with speech, and in content between NSs and NNSs. 


3. The use of these nonverbal behaviors by NNSs can differ 
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ny 


according to oral L2 proficiency. 
g 


da 
. 


Degrees of nonverbal ability can be identified and applied to 
a meaningful rating system. 


As a pilot study, there are limitaticns to the scales 


‘outlined here. Only three behaviors are apeliea here. This study 


dig nol include horizontal head movement (shake), facial 
expression, posture, or interpersonal distance. The three 
behaviors chosen, however, can be easily observed by raters. These 


scales are meant for research purposes, therefore only one cultural 
group was used as a control for data collection. Much work remains 
to be done before this can be applied outside of research, 
especially in the area of norms. 

The next step is to test the test. By applying these scales 
to the NS/NNS data in this study using independent raters, it will 
bo possible to answer the questions of whether the scales are 
reliable and whether nonverbal ability is an independent construct 


from linguistic and oral L2 proficiency. 


32 





Assessing Nonverbal Ability 
32 


References 

Argyle, M. (1988). Bodily Communication, 2nd ed. Madison, Conn.: 
International Universities Press. 

Bachman, L.S. (1988). Problems in examining Lhe validily of the 
ACTFL Oral Proficiency Interview. Studies in Second Language 
Acquisition, 10, 149-163. 

Bachman, L.S. (1990!. Fundamental Considerations in Language 
Testing. Oxtord: Oxford University Press. 

Bachman, L.S. & Palmer, A.S. (1982). The construct validation of 
some components of communicative proficiency. TESCL Quarterly, 
16(4), 449-465. 

Bachman, L.S. & Palmer, A.S. (1983). Oral Interview Test of 
Communicative Proficiency in English. Urbana, Ill.: 
Photo-offset. 

Bachman, L.S. & Savignon, S. (1986). The evaluation of 
communicative language proficiency: A critique of the ACTFL 
oral interview. The Modern Language Journal 70(4), 380-390. 

Baird, L.L. (1983). The search for communication skills. 
Educational Testing Service Research Report, No. 83-14. 

Bialystok, E. (1990). Communicalion Stralegies: A Psychologicel 
Analysis of Second-language Use. Oxford: Basil Blackwell. 

Bialystok, E. & Sharwood Smith, M. (1985). Interlanguage is not a 


state of mind: An evaluation of the construct for second 


ERIC 33 


JA Fun Text Provided by ERIC 








Assessing Nonverbal Ability 
33 


language acquisition. Applied Linguistics 6, 202-7. 

Brow, J.D. (1992). Testing in Language Programs. Unpublished 
Manuscript. Honolulu, HI: University of Hawaii at Manoa. 

Canale, M. (1983). From communicative compelence to comeunicalive 
language pedagogy. In J. Richards & R. Schmidt (Eds.). 
Language and Communication. London: Longman. 

Canale, M. (1988). The measurement of communicative competence. 


Annuai Review of Applied Linguistics, 8, 67-84. 





Canale, M. & Swain, M. (1980). Theoretical bases of communicative 
approaches to second-language teaching and testing. Applied 
Linguistics, 1, 1-47. 

Du Bois, J.W., Schuetze-Coburn, S., Cumming, S., & Paolino, D. 
(1993). Outline of discourse transcription, In J.A. Edwards & 
M.D. Lampert (Eds.). Talking Data: Transcription and Coding in 
Discourse Research. Hillsdale, NJ: Lawrence Erlbaum. 

Duncan, S., Jr. and Fiske, D.W. (1985). Interaction Structure and 
Strategy. Cambridge, England: Cambridge University Press. 

Edmondson, W. (1981). Spoken Discourse. New York: Longman. 

Educational Testing Service. (1992). TOEFL Test and Score Manual, 
1992-93 Edition. Princeton, NJ: Educational Testing Service. 

Educational Testing Service. (1990). Test cf Spoken English: Manual 
for Score Users. Princeton, NJ: Educational Testing Service. 


Ekman, P. (1976). Movements with precise meanings. Journal of 


(e 
ERIC 











O. 
ERIC 





34 


Communication. 3, 14-726. 

Færch, C. & Kasper, G. (1984). Two ways of defining communication 
strategies. Language Learning, 34(1), 45-63. 

Ferch, C., Haastrup, K. & Phillipson, R. (1984). Learner Language 
and Language Learning. Clevedon, Avon: Multilingual Matters. 

Haastrup, K. (1986). Pragmatic and strategic competence in the 
assessment of oral proficiency. System i4(1), 71-79. 

Kasper G. (1981). Pragmatische Aspekte in der Interimspracne. 
Tuebingen: Narr. 

Kasper, G. (1992). Pragmatic transfer. Second Language Research 
8(3), 203-231. 

Kendon, A. (1984). Some uses of gestures. In D. Tannen & M. 
Savilie-Troike (Eds.). Perspectives on Silence. Norwood, NJ: 
Abiex. 

Kendon, A. & Cook, M. (1969). The consistency of gaze patterns in 
social interaction. British Journal of Psychology, 69(4), 
481-94. 

Knapp, M.L. & Hall, J.A. (1992). Nonverbal Communication in Human 
Interaction, 3rd ed. New York: Holt Rinehart and Winston. 

La France, M. & Mayo, C. (1978). Gaze direction in interracial 
dyadic communication. Ethnicity, 5, 1€7-73. 

Maynard, S.K. (1987). Interactional functions of a nonverbal sign: 


Head movement in Japanese dyadic casuai conversation. Journal 








Assessing Nonverbal Ability 


35 


or Pragmatics, 11, 589-606. 
Maynard, $.K. (1989). Japanese Conversation: Self-contextualization 


through Structure and Interactional Management. Norwood, NJ: 


Maynard, S.K. (1990). Conversation management in contrast: Listener 
response in Japanese and American English. Journal of 
Pragmatics, 14, 397-412. 

McNeill, D. (1985). So you think gestures are nonverbai? 
Psychological Review, 92, 350-371. 

Nambiar, M.K. & Goon, C. (1993). Assessment of oral skills: A 
comparison of scores obtained through audio recordings to 


those obtained througn face-to-face interaction, RELC Journal, 


Neu, J. (1990). Assessing the role of nonverbal communication in 
the acquisition of communicative competence in L2. In R.C. 
Scarcella, E.S. Andersen, & S.D. Krashen (Eds.). Developing 
Communicetive Competence in a Second Language. New York: 
Newbury House. 

Rubin, R.B. (1982). Assessing speaking and listening competence at 
the college level: The communication competency assessment 
instrument. Communication Education, 31, 19-32. 

von Rkaffler-Engel, W. (1980). Kinesics and paralinguistics: A 


neglected factor in second-language research and teaching. 


o 36 
ERIC 








Rssessing Nonverbal Ability 
6 


Ww 


Canadian Modern Language Review, 362), 225-37. 
White, S. (1989). Backchannels across cultures: A study of 


Americans and Japanese. Language in Society 18, 59-76. 


Q 
ERIC 








Q 


ERIC 


JA Fun Text Provided by ERIC 





Assessing Nonverbal Ability 
37 


Appendix A — English Versions of Role Plays 


Role Play Instructions 


Role Play 1 


Role A: 


Role B: 


Role Play 2 


Role A: 


Role B: 


You will be asked to perform a number of role plays. 
Read each situation and imagine that you are really the 
person in the role play. Specific details are not 
given, but try to speak in as much as detail as 
possible. 

(Warm-up) 

You are at the school cafeteria talking to a friend who 
is very good at Japanese. You want to borrow some 
books from a friend so that you can study Japanese. 

You are at the school cafeteria talking to a friend who 
wants to study Japanese. You are very good at Japanese 
so your friend wants to borrow some books from you to 
study. Offer your friend some books and try to give 


some advice for studying Japanese. 


You are a student applying to enter an English language 
school. You are being interviewed by one of the 
teachers. Answer the teacher's questions about 
yourself, your interests, and your reascns for studying 
English in as much detail as possible. 

You are a teacher at an English language school. You 


must interview an applicant to evaluate his speaking 


Q9 
ea) 








Role Play 


Role A: 


Role B: 
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ability. First, ask the applicant basic background 

questions (name, birthplace, etc.). Then, ask about 

interests, language learning experience, and reasons 
for studying English. Ask in as much detail as 


possible. 


You are the manager of a convenience store. You 
must interview a student for a part-time job. 
First, ask the applicant basic background 
questions (name, birthplace, etc.). Then, ask 
about interests, working experience, and reasons 
for wanting to work at your store. Ask in as 
much detail as possible. 

You are student applying for a part-time job at a 
convenience store. You are being interviewed by 
the manager of the store. Answer the manager's 
questions about yourself, your interests, and 
your reasons for wanting a job at the store. 


Answer in as much detail as possible 


(Role Play 4 is performed individually by both subjects with a 


Role Play 4 


Role A/B: 


native speaker) 
— Non-native Speaker 
You want to apply to an American university's graduate 


school to study in your major. You need a letter of 
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recommendation from your teacher. You go to your 
teacher's of:ice to ask him for one. Be ready to 
explain why you need the letter, what information it 
should contain, when you need it, and what your teacher 
should do with it in as much detail as possible. 

- Nalive Speaker 

You are a professor at a Japanese university. A 
student comes to you to ask for a letter of 
recommendation so that i can study at an American 
University. Ask the student why he wants *a study 
abroad, what information the letter should contain, 
when he needs it, and where he should send it. Try to 
get the student to explain in as much detail «4s 


possible. 


Instructions for NS Assistant 


Ta Greet 


cee) 


mouth. 


the student: "Hi, (name), what can I do for you?" 


2. Remember. You do not know what the student wants beforehand. 


Help the student speak, but do not put words in his/her 


Th: burden to communicate should be on the student as 


long as pauses are not too long. Be patient. 


4, If the student does not give instructions about the letter of 
recommendation, ask questions about why he/she wants to study 
abroad, the information to be contained in the letter, when 


it is needed, where to send it, and so on. 
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time, close the 


word of “good luck." 
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Appendix B - Transcription Symbols 


Discourse Transcription (Du Bois, et. al., 1993) 


Intonation unit 
Truncated intonation 
unit 
Word 
Truncated word 
Speaker identity/ 
turn start 
Speech overlap 
Transitional Continuity 
Final 
Continuing 
Appeal 
Pause 
Long 
Medium 
short 
Latching 
vocal Noises 


Vocal noises 


Laughter 
Tra scribers Perspective 
Researcher's comment 
Uncertain hearing 
Indecipherabie syllable 
Specialized notation 


Nontranscription line 








{soace} 


THROAT, GULP, SWALLOW, 


SNIFF, COUGH, TSK (tongue 


ei ek) 


e 
v 
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hesitation 


backchannel 


aittirmation 


negative bc 
Nonverbal Transcription 


Role identity 


Nonverbal notation 
Hand gestures: 
Head Movement 


Head nods: 


Gaze in direction 


of interlocutor 


tbc} 


Gaze away from 


interlocutor 


uh, unk, um, er 

hm, m, huh, hunh 

mhm, unkhurh, uhuh, oh, 
ah 


unh-unh 


v 


b) (subject B: symbols are 


G ("brief description") 


| 
A 
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Footnotes 


All examples are preceded by the number of the example, the 
title of the role play, and the subject number preceded by NS or 
NNS. Coding is made at the approximate onset of each occasion. 
Duration is not included. Nonverbdal notes apply to the subjects 
for listening and speaking and are not included for the NS 
assistant's behavior. See Appendix B for norverbal transcription 


symbols. 
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Table 1 Descriptive statistics for proficiency tests 
t 


TOEFL SPEAK 


Pel Pt.2 . Pt. 3. Total . Pron. Grammar. Fluency . Comprehensibility 


N =28 
Mean 41.07 39.46 40.43 403.21 1.15 101 107 (111.21 
SD 4.77 8.07 7.52 59.66 49 65 44 46.30 


Low-high 32-52 26-57 28-60 310-553 0.3-2.2 0-2.3  0.3-2.15 30.5-210 
T, -- -- -- 93 


re 94 97 94 98 
SEM Se artes, ae IS on 15.78 __ „12 eee etl So ae eal oes 6.55 CEREN 


* Corrected with the Spearman-Brown prophecy formula 


for nonverbal behaviors in role plays 


ct 
A 
th 


Table 2 Descriptive statisti 





Essa ome Se ee Go ee 
Role Play: 2 3 34 4 2 3 3/4404 2 3 3/4 
Native Speakers (N = 20) 
Mean 20.550 21.40 20.975 15.200 21.45 21.750 21.600 24.800 3.550 3.200 3.375 
SD 6.461 11.04 6.290 6.202 12.65 9.767 8.433 7.527 3.735 3.458 2.687 
Low-high 8-35 7-43 7.5-33 6-29 4-51 6-44 8-39 14-48 0-13 0-13 0-8 
Range 28 37 26.5 24 48 39 32 35 14 14 8 
Nonnative Speakers (N = 28) 
Mean 21.750 23.390 22.571 30.89 17.710 18.500 18.107 23.320 2.107 2.714 2.411 
SD 8.374 9.645 8.525 11.98 6.943 5.137 5.547 6403 2.87 2.788 2.560 
Low-High 4.34 5-38 7.5-36 14-61 7-37 10-30 85-33.5 14-37 0-9 0-10 0-9.5 

10.5 


Range 31 34 28.5 48 31 21 26 25 10 11 
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4.179 
4.660 
0-18 

19 
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i.) Native speakers 
Mi Non-native speakers 


5 o os 





gg ` i 


Nods 2/3 Nods 4 Gaze 2/3 Gaze4 Gesture 2/3 Gesture 4 





Nonverbal behaviors by role play and group 


Figure 1 Nonverbal behaviors in role plays 


Communicative Language Ability 


A Language competencies 








1 Grammatical competence Nonverbal ability 


2 Textual competence Textual ability 





3 Hiocutionary competence 
4 Sociolinguistic competence 


B Strategic competence 





Strategic ability 


C Psychophysiological mechanisms 


Figure 2 Communicative Language Ability (CLA) (Bachman, 


1990) with a nonverbal component 
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Scales of nonverbal ability (NOVA) 


Textual Ability 


Rating 


o> 


ray 


nN 


tad 


Frequency 

Extremely limited use of 
head nods and infrequent 
changes in gaze direction 
toward partner in 
conversation 

Frequent use of head nods 
and changes in gaze 
direction that are not 
acceptable by native speaker 
norms 

Frequency of head nous and 
changes in gaze direction 
approaches native speaker 
norms 

Frequency of head nods and 
changes in gaze direction 
acceptable by native speaker 


norms 


48 


Rating 


Appropriateness 
Totally inappropriate use of 
head nods and gaze direction 


by native speaker norms 


Frequent inappropriate use 
of head nods and changes in 


gaze direction 


Rare inappropriate use of 
head nods and changes in 


gaze direction 


Use of head nods and changes 


in gaze direction acceptable 


by native speaker norms 


(Table Continues) 








Assessing Nonverbal Ability 
48 


Strategic Ability 


Rating 
Compensatory Usage 

0 No evidence of hand gestures 
to solve considerable 
linguistic problems 


l Limited use of hand gestures 


cr 


o solve linguistic problems 
with occasionally 


unsuccessful results 


2 Hand gestures successfully 
used to solve linguistic 
problems 

3 Rare linguistic problems 
requiring the use of hand 


gestures for compensation 


49 





Rating 


Appropriateness 
Never uses hand gestures to 


support or enhance meaning 


Occasionally uses hand 
gestures to support or 
enhance meaning, often 
inappropriateiy by native 
speaker norms 

Use of hand gestures 
approaches native speaker 
norms 

Use of hand gestures 


appropriate by native 


speaker norms 





